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Abstract 



The larse number of rear end collisions due to driver inattention has been identified as a ™ a j° r ™<°- 
avoidance tests have also been performed on test tracks. 



1 Inttrodliuicition 

One of the major challenges of the next generation of road transportation vehicles is tc 
nrooortion of the total accidents (29.5% in USA and 28% in Germany)[l]. 

^attention by the driver is identified as me cause for 91% ^^^^^^l 
to a 1992 study by Daimler-Benz (cited in [1]), if car drivers have a ^ 5 : second f^^^Zf^ t 
about 60% of rear-end collisions can be prevented. An extra second of warning tune can prevent about 

52 pLr^SLon Warning (FCW) high on the list of solutions that ca^^ 
2£& to reduction of the number and the severity ^^^^ A J5TS5TS^ 
the vehicle could provide a practical solution to this V^^^^^^^^ (narrow field 
systems available today (typically based on Radar sensors) and their limited From a 
of Set and poor latemi resolution) have prevented such systems from e ^™!* e ^ta system 
technological point of view, fusion of radar and vision is an attractive approach. In such system 



(a)system components 




(b)display panel 
Figure 1: The MobilEye-AWS. 



the radar gives accurate range and range-rate measurements while vision solves the angular accuracy 
problem of radar. However this solution is costly. 

™ S oaner presents MobilEye's vision based FCW system including experimental resu ts. The al- 

S-peoaTor brle-pedal position etc.) are not required but can be used m the complete system as a 
Kondary filter to further anticipate the drivers intentions and reduce unnecessary alarms. 

1.1 The MobuEye Advance Warning System (AWS) 

Using a single forward facing camera located typically near the rear view mirror the MobilEye-AWS 
detect id iacks vehicles on the road ahead providing range, relative speed and lane position data 
S system d^s also the lane markings and road edges and measures the distance of the host vehicte 
to road boundaries. Thus it combines together on the same platform F «7^ Coll ^^" S ' ^ 
Departure Warning and Headway Monitoring. It can also be connected to active safety systems^ 
Se camera usedtn this work has VGA sensor (640 x 480). and a horizontal FOV of 47° A smal 
Splay ^ut and a pair of left and right speakers (see Figure 1) inside the car provide audio 
wSgTallowing the driver to react to various types of dangerous situations and to reduce the risk of 
accidents. The system warning thresholds are adaptable to different driving styles. 



2 Momentary Time to Contact 

One method for FCW analyzed I. j[4] I uses Hne »~~~<^«2Z JLTSt^SS 
Collision Warning (FCW) is issued when the time-ta-contact^TIC) is lower man a 
typically 2 seconds. We will define the momentary Time To Contact (T m ) as 



r CD 



where Z is the relative distance to the target and V the relative velocity. 

S^e Isince and relative speed are not natural vision measures, we will show ^ that .we cm represen 
Te momentary TTC as a function of scale-change in the image in a givensamplmg mterval (A,). This 
value can be computed accurately from the image sequence as shown in [2J. 
The perspective projection model of the camera gives: 



fW (2) 



W > = Z, 

where w, is the width of the target in the image at time t, Z, is the distance to the target, W is the vehicle 
width, and / is the camera focal length. ~__^ c 
We define scale-change S as the ratio between the width in the image in two consecutive frames 

c_ w ' -5r_^ (3) 

When the time interval between Zi and Z 0 is small we can write: 



Zj = Z 0 + VAl (4) 

Thus: Zi (5) 

Extracting f - from the equation above yields: 

Z± A/ (6) 
V S-l' 



Equation 6 gives the momentary TTC solely on scale-change and time information. 

3 Modeling Acceleration 

The problem with the momentary TTC computation is that it neglects relative acceleration 1 be^een 
me two vehTcles. Relative acceleration will occur when the target vehicle P** 0 ™*™? 6 ™*^™ 
wheTme host vehicle is slowing down to avoid collision. Both cases are very important m an FCW 
rppl^ Not detecting theLst vehicle slowing down will give many false alarms (e.g. when 



r 



nearing a stop light). Using the brake signal may not be enough since many times the driver might slow 

down bv simply taking his or her foot off from the gas pedal. 

T^g IntoTccount relative acceleration, the relative distance between the two velucle as a funct.cn of 

time is given by: j m 

TTC (which we will denote simply as T) is the time thatZ = 0( as in [4]), thus: 



As mentioned above, the values distance, speed and acceleration are not "f^^f^^^ 
in the image. 

The momentary TTC is given by : z 

Thus its derivative is: -ZV + VZ (10) 

Since Z=V and V = a and we get: 



-»* + q-Z _ oj = ^Z t (11) 
Let us define an assistance variable . a • Z (12) 

C = 2m + 1 — 

The r m and its derivative are measures taken from the current image thus Z and V actually refers to Z 0 
and V 0 , so we can say: Zq 

Tm = ~~Vo 

and aZp (14) 

Extracting a from (14) we get: 

Now we'll show how to define T in the vision natural measures T m and t m . Substituting (15) in (8) we 
g et: 

-v ^jvg+ic-vz (16) 

z ^ L= f L K 0 V1+2C (17) 
" " a 
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Substituting (15) in (17) results in: 



-v 0 +v 0 -yi+2c (18) 

T = —ft 



L Zo 



-1+V1+2C (19 ) 

Substituting -T m for (equation 13) we get: 

-1 + V1+2C (20) 
T = - ~c~ 

— Tm 



-l+vT+2C (21) 

— J-m ~F< 



l-vTT2C (22) 



where C is a function of t m as in (12). 

4 Lateral Collision Decision 

After computing the TTC and determining that we are rapidly closing the distance to the target the 
After computmg me ii ^ to determine if we are in fact on a possible collision course. 

whhout knowing the distance to the target vehicle or the target vehicles physical width. We wnl then 
show how to integrate the range information if it is known. 

Let*,(rt andx r (/) be the image coordinates of the left and right edges of the target vehicle at tm*'- Le* 
Z?0) be the dlSance at time ? = 0 and we set Z(0) = 1 in some arbitfary units Usmg the perspective 
projection equation we can compute the vehicle width in the same arbitrary units: 

(x r (0)-x,(0))Z(0) (23) 
fV = j 

As we approach the followed vehicle we can compute the range Z(t): 

z(t) - f* (24) 

(x r (0)-x,(0))Z(0) (25) 
x r {t)-xi{t) 
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(a) 




0>) 

F5OTirP 7- Results from balloon-car experiments. Lateral position of the outside edges of the target ve- 
H^dlnn^^apP^^ouJas a Junction of time (t). TTC is indicated by the tall ^ttcalUne 
an^hoZe^cTfposiZn is miked by an X. (a) collision course, (b) one of the vehicles lS perfo^ng 
an avoidance maneuver. 

We can then use Z(f ) to compute the relative lateral position of the car: 



MO = 



X r {t) = 



x,{t)Z{t) 
f 

x r (t)Z(t) 
f 



(26) 
(27) 



Substituting (24) into the above we get: 

Xiif) = 

Xr{t) 



x,(t)Z(0)x r (0)-xi(p) (28 ) 

/ x r (t)-Xi(t) 

xM)Z(0) xr(0)-xi(0) (29) 
- / x r (t)-xi{t) 

Figure 2a S how S meresultsoftr« 

of time. These lines are then extrapolated to time t = TTC W » "J" » ™ " ^ ^ en ^ 
the right then the target vehicle is on a collision course. If both*, W "^^^^iff taSiwn 
targefvehicle in not on a collision course wi* *™^^^t?r 
in figure 2b. The experiments were performed on a test track with a Daiioon car as, 
We Tse the last 9 measurements (0.9 sec at 10 HZ) and perform a linear fit. Since ^e only need to apply 
this procedure when the TTC is small (T < 3sec~) the extrapolation error is not large. 



(a) 

Figure 3: The remote mounting structure. 

Note that the lateral positions JQ(t) and X r {t) are in our arbitrary units. If we have even « 

of Z(0) then we convert the lateral position to meters and create a safety margm around the host vehicle. 

A method for computing the range estimate is shown in [2] 

5 Experiments and Results 
5.1 Test Apparatus 

in normal driving conditions FCW rarely occurs and only false alarms can be tested in this way. In 
order to test FCW reliability and accuracy specific tests were performed in a test area. 
The tests had to simulate car crash, for this purpose a lightweight, rigid metal s^ure was bu^and 
was assembled on the top of the host vehicle (a Renault Kangoo), as shown in figure 3a. The camera 

hiffher than it would be if mounted inside the host vehicle. . 

Fte^ TLws a crash simulation sequence. In order to know exactly the ^.**«* *" ^ 

the image (see figure 4d). 
5.2 Accuracy Tests 

The first set of tests was TTC accuracy. An accuracy test was made by ^corftag £ ^rio»q^ 
and then analyzing the TTC signal offline in the office. In this analysis the collision time is known and 
we can compare the predicted time to actual collision time. 

There were two different scene dynamics measured: constant speed and relative acceleration. 

1 , Constant Relative Speed: the host vehicle drives toward a stationary target vehicle and past it at 
a constant speed. 





Figure 4: The camera passing just over the top of the target vehicle as 
simulation. In (d) the camera is exactly above the rear bumper 



recorded during a collision 



Constant Spood Scenarios 
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Actual Torco To CoflisJon 



estimated TTC (Y-axis) for all ten sequences. 

the stopping (or stopped) lead vehicle. 

estimated TTC for all ten cups k & A { increasing when the TTC is 

the estimated TTC is reasonably accurate below 2 seconds wun m 

lar ? r " * th. results from one sequence from scenario 2: lead vehicle decelerating. We see 

In Figure 6 we see the results from usi a constant acce l e ration model is 

^;T;S^e,t line. In Figure 7 we see that this behavior is qualitatively 
the same for all the ten sequences. 



Relative Acceleration Example 



Actual Ttmo To Collision 



Figure 6: A single example of scenario 2: lead vehicle is decelerating. Comparing momentary TTC 
(crosses) and the modified TTC (triangles) to the ground truth TTC. 



Acceleration Scenarios 



1 




" Actual Time To Collision 

Figure 7- 5ce W «r/o 2: fe«rf ve/»cfe is decelerating. Comparing momentary TTC (crosses) and the 
modified TTC (triangles) to the ground truth TTC for all 10 clips. 

In order to see quantitatively how the errors in TTC estimates depend on fee actual TTC we binned 
Se melsSemeSto 1 second bins and computed the mean error and standard delation for each bm 
Sparely Th^ Results are shown in table 1. We see that for larger actual TTC the standard dev ation 
of mfesLated ^TC increases. For TTC values below 2 seconds the TTC is quite accurate even in the 
£ete„2 of ^acceleration. Future work involves improving the tracking accuracy (namely scale 
measurement) to allow for reliable warnings at TTC of 3 seconds and up. 

5.3 Warning Tests 

in addition to measuring the signal quality, a series of tests was performed ^^^T 
of the system. There were three types of scenarios: sudden brake avoidance, slow avoidance (slowly 
stopping behind a parking vehicle) and lateral avoidance. 
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1 TTC (sec) | Constant Speed | Acceleration 





mean 


std 


mean 


std 


0-1 


0.01 


0.046 


0.002 


0.039 


1-2 


-0.05 


0.022 


-0.042 


0.26 


2-3 


-0.07 


0.54 


-0.37 


1.22 


3-4 


0.087 


0.76 


-0.7 


2.83 


4-5 


-0.52 


1.03 


N/A 


N/A 



^ u, i c ~ ,„ TTC estimates for the two tests scenarios: constant speed and relative accelera- 

computed for each bin. 

5.3.1 Sudden Brake Avoidance 

5SS both vehicles were stopped was 2m and the minimum distance lm. . ... 

rather than serious injury. 



5.3.2 Slow Avoidance 
5.3.3 Lateral Avoidance 

The host vehicle approaches the lead vehicle at a constant speed and when TTC i. > three > second s the 
SstveWcTdriver switches to the next lane to avoid crash. This is the typical «^on of the host 
veWcTe do ing a very tight passing maneuver. This test was done with normal mounted camera inside 
^^\^tbf*« mouft. in none of the ten tests a false warning was g,ven. 



6 Summary and Future Work 

i c<-\v w«tf. m w h\ch uses a single camera as input. Image scale change and image 
We have presented an FCW system "J^™^ targe t vehicle is on a collision path thus 

position are used directly for computing TTC and whe ther tb e ^"J* real-time hardware 



desired. 
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